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CLAIMS: 

1 . A method for encoding a video frame taking a human visual 
system into consideration, said method comprising the steps of: 

(a) increasing quantization in sectors of said video frame where 
quantization noise and coding artifacts are less noticeable to said 
human visual system; and 

(b) decreasing quantization in sectors of said video frame where 
quantization noise and coding artifacts are more noticeable to said 
human visual system. 

2. The method of claim 1 ? comprising: 

(a) said step of increasing quantization freeing surplus bits; and 

(b) said step of decreasing quantization requiring additional bits. 



15 3. The method of claim 2, said step of decreasing quantization using 

said surplus bits to realize the additional bits required. 

4. The method of claim 3 further comprising the step of maintaining 
uniform quantization in said video frame if said step of increasing quantization and said 

20 step of decreasing quantization would require more bits than said step of maintaining 
uniform quantization. 

5. The method of claim 3 further comprising the step of maintaining 
uniform quantization in said video frame if said surplus bits are insufficient to realize said 

25 additional bits required by said step of decreasing quantization. 

6. The method of claim 3 further comprising the step of maintaining 
uniform quantization in said video frame if said step of increasing quantization and said 
step of decreasing quantization are less efficient than said step of maintaining uniform 

30 quantization. 
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7. A method for adaptively quantizing the encoding of a video frame, 
said method comprising the steps of: 

(a) selecting a first default QP value, a second higher QP value higher 
than said first default QP value, and a third lower QP value lower 
than said first default QP value; 

(b) predicting a baseline number of bits to encode said video frame 
using said first default QP value; 

(c) classifying portions of said video frame into busy sectors and flat 
sectors; 

(d) predicting whether said video frame has sufficient busy sectors to 
produce surplus bits over said baseline number of bits if said busy 
sectors are encoded using said second higher QP value; 

(e) predicting whether said surplus bits are sufficient to encode said 
flat sectors using said third lower QP value; 

(f) quantifying said busy sectors using said second higher QP value 
and said flat sectors using said third lower QP value if said video 
frame has said sufficient busy sectors and said surplus bits are 
sufficient to encode said flat sectors; and 

(g) encoding said video frame entirely using said first default QP value 
if encoding said video frame entirely using said first default QP 
value would be more efficient than quantifying said busy sectors 
using said second higher QP value and quantifying said flat sectors 
using said third lower QP value. 

8. The method of claim 7 further comprising the step of repeating 
steps (a) - (g) for each of a plurality of video frames. 
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9. The method of claim 7, wherein said step of encoding said video 
frame entirely using said first default QP value is more efficient than said step of 
quantifying said busy sectors using said second higher QP value and said step of 
quantifying said flat sectors using said third lower QP value if said video frame does not 

5 have said sufficient busy sectors. 

10. The method of claim 7, wherein said step of encoding said video 
frame entirely using said first default QP value is more efficient than said step of 
quantifying said busy sectors using said second higher QP value and said step of 

1 0 quantifying said flat sectors using said third lower QP value if said surplus bits are 
insufficient in number to encode said flat sectors. 



1 1 . The method of claim 7, wherein said step of predicting whether a 
video frame has sufficient busy sectors further comprises the step of setting a criterion 

1 5 that said video frame does have sufficient busy sectors if a predicted number of bits that 
would be required to encode all said flat sectors of said video frame using said third lower 
QP value is less than or equal to a predicted number of said surplus bits that would be 
provided by encoding all said busy sectors of said video frame using said second higher 
QP value. 

20 

12. A method for adaptive quantization of video encoding based on 
prediction of required bits, said method comprising the steps of: 

(a) providing a video frame; 

(b) establishing a uniform QP value for said video frame; 

25 (c) classifying said video frame into busy sectors, normal sectors, and 

flat sectors; 

(d) decreasing quantization below said uniform QP value in said flat 
sectors of said video frame that can be encoded with a predicted 
relatively small increase in said required bits needed to encode said 
30 flat sectors using a QP value below said uniform QP value; 
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(e) increasing quantization above said uniform QP value in said busy 
sectors of said video frame only when a predicted decrease in said 
required bits needed to encode said busy sectors using a QP value 
above said uniform QP value is relatively large; and 
5 (f) reverting to said uniform QP value for all sectors of said video 

frame if a predicted number of said required bits for encoding all 
said flat sectors in said video frame using said QP value below said 
uniform QP value would be greater than a predicted number of 
surplus bits provided by encoding all said busy sectors in said 
10 video frame using said QP value above said uniform QP value. 

13. The method of claim 12 further comprising the step of repeating 
steps (a) - (f) for each of a plurality of video frames. 

15 14. The method of claim 12, said step of classifying said video frame 

into busy sectors, normal sectors, and flat sectors further comprising the step of 
classifying visual textures using high, medium, and low texture categories and a predicted 
required bits value for at least a portion of each sector in said video frame. 

20 15. The method of claim 12 further comprising the step of summing an 

energy value for at least a portion of each sector in said video frame. 

16. The method of claim 14, said step of classifying visual textures 
further comprising a step selected from the group of steps consisting of: 
25 (a) calculating said energy value for said at least a portion of each 

sector using at least one variance value; 

(b) calculating said energy value for said at least a portion of each 
sector using at least one luminance value; and 

(c) calculating said energy value for said at least a portion of each 
30 sector using at least one activity value. 
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17. The method of claim 14, said step of classifying visual textures 
further comprising the step of calculating a predicted required bits value for each sector 
using a prediction error energy value. 

5 

18. The method of claim 12 further comprising the step of decreasing 
said QP by a first constant. 



19 The method of claim 12 further comprising the step of increasing 
1 0 said QP by a second constant. 



20. A method for adaptive quantization of the encoding of a video 
frame based on predicted bit rate, said method comprising the steps of: 

(a) decreasing a QP in low-textured sectors of said video frame that 

1 5 can be encoded with a relatively small increase in said predicted bit 

rate; 

(b) increasing a QP in high-textured sectors of said video frame only 
when a decrease in said predicted bit rate is relatively large; and 

(c) turning off said steps of decreasing a QP and increasing a QP if 
20 said relatively small increase in said predicted bit rate for encoding 

said low-textured sectors is larger than said decrease in said 
predicted bit rate for encoding said high-textured sectors. 



21 . The method of claim 20, said steps of decreasing a QP and 
25 increasing a QP further comprising the step of increasing and decreasing a QP by a 

constant. 

22. The method of claim 21 further comprising a step selected from the 
group of steps consisting of: 
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(a) selecting a constant of approximately 2.0 for linear quantization; 
and 

(b) selecting a constant of approximately 1 .5 for nonlinear 
quantization. 

5 

23. The method of claim 20, further comprising the steps of: 

(a) providing a complete digital representation of at least one video 
frame encoded by said adaptive quantization to a decoder; and 

(b) displaying said at least one video frame on a display device in a 
10 decoded form. 

24. A method for determining the rate of data compression for digital 
storage of a sequence of video frames, said method comprising the steps of: 

(a) dividing said sequence of video frames into individual video 
15 frames; 

(b) establishing a usual rate of data compression for each video frame; 

(c) dividing each video frame into sectors; 

(d) predicting a bit rate cost for digitally storing each sector; 

(e) predicting a visual complexity for each sector; 

20 (f) dividing said usual rate of data compression by a first constant for 

each sector in which said predicted visual complexity is low and 
said predicted bit rate cost for digitally storing is low; 

(g) multiplying said usual rate of data compression by a second 
constant for each sector in which said visual complexity is high 

25 and said predicted bit rate cost for digitally storing is high; 

(h) maintaining said usual rate of data compression at a nearly 
constant value for all sectors in said video frame if an absolute 
value of a predicted increase in an overall bit rate cost for digital 
storage of all sectors in which said predicted visual complexity is 

30 low and said predicted bit rate cost for digitally storing is low is 
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greater than an absolute value of a predicted decrease in an overall 
bit rate cost for digital storage of all sectors in which said predicted 
visual complexity is high and said predicted bit rate cost for 
digitally storing is high; and 
(i) encoding each sector of each video frame in said sequence of video 
frames to obtain a complete digital representation of said sequence 
of video frames. 

25. The method of claim 24 further comprising the steps of: 

(a) providing said complete digital representation of said sequence of 
video frames to a decoder; and 

(b) displaying said sequence of video frames on a display device in a 
decoded form. 

26. The method of claim 24, wherein said step of predicting a bit rate 
cost for digitally storing each sector further comprises the steps of calculating said bit rate 
cost by: 

(a) determining an energy parameter for each sector; 

(b) determining a quantization step size value for each sector; 

(c) defining each sector having the square root of a value of said 
energy parameter less than said quantization step size value 
divided by a third constant as having a low bit rate cost for digital 
storage; and 

(d) defining each sector having the square root of a value of said 
energy parameter greater than said quantization step size value 
multiplied by a fourth constant as having a high bit rate cost for 
digital storage. 
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27, The method of claim 24, said step of predicting a visual 
complexity for each sector further comprising the steps of calculating said visual 
complexity by: 

(a) determining an activity parameter for each sector; 
5 (b) determining first and second threshold activity values for 

categorizing said visual complexity of each sector; 
(c) defining each sector having a value of said activity parameter less 

than or equal to said first threshold activity value as having low 

visual complexity; 

10 (d) defining each sector having a value of said activity parameter both 

greater than said first threshold activity value and less than or equal 
to said second threshold activity value as having normal visual 
complexity; and 

(e) defining each sector having a value of said activity parameter 
1 5 greater than said second threshold activity value as having high 

visual complexity. 

28. The method of claim 24, said step of predicting a visual 
complexity for each sector further comprising the steps of calculating said visual 

20 complexity by: 

(a) determining a maximum activity parameter for each sector; 

(b) determining a minimum activity parameter for each sector; 

(c) determining first and second threshold activity values for 
categorizing said visual complexity of each sector; 

25 (d) defining each sector having a value of said maximum activity 

parameter less than or equal to said first threshold activity value as 
having low visual complexity; 
(e) defining each sector having a value of said maximum activity 
parameter greater than said first threshold activity value and a 

30 value of said minimum activity parameter less than or equal to said 

34 PDXDOCS: 12 77552.2 



Express Mail No.: EI618336324US 



5 



CD 



second threshold activity value as having normal visual 
complexity; and 

defining each sector having a value of said minimum activity 
parameter greater than said second threshold activity value as 
having high visual complexity. 



29. The method of claim 24, said step of maintaining said usual rate of 
data compression further comprising the step of calculating said increase in overall bit 
rate cost by summing said bit rate costs for each sector in which said predicted visual 

10 complexity is low and said predicted bit rate cost for digitally storing is low. 

30. The method of claim 24, said step of maintaining said usual rate of 
data compression further comprising the step of calculating said decrease in overall bit 
rate cost by summing said bit rate costs for each sector in which said predicted visual 

1 5 complexity is high and said predicted bit rate cost for digitally storing is high. 

3 1 . The method of claim 24, said steps of dividing said usual rate of 
data compression by a first constant and multiplying said usual rate of data compression 
by a second constant further comprising the step of executing linear quantization. 



32. The method of claim 24, said steps of dividing said usual rate of 
data compression by a first constant and multiplying said usual rate of data compression 
by a second constant further comprising the step of executing nonlinear quantization. 



data compression further comprising the step of filtering values of a ratio formed between 
said sectors with low visual complexity and said sectors with high visual complexity for 
each video frame so that said sequence of video frames will maintain said usual rate of 
data compression despite isolated fluctuations in said ratio for individual video frames. 



20 
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33. The method of claim 24, said step of maintaining said usual rate of 



30 



35 



PDXDOCS: 11 77552.2 



Express Mail No.: EI618336324US 
34. An encoder for coding a digital picture in a video frame, 

comprising: 

(a) a frame divider for dividing said video frame into sectors; 

(b) a bit rate predictor for predicting a bit rate cost for digitally storing 
5 each sector; 

(c) a visual complexity predictor for predicting a visual complexity for 
each sector; 

(d) a quantization divider for dividing a usual rate of data compression 
by a first constant for each sector in which said predicted visual 

1 0 complexity is low and said predicted bit rate cost for digitally 

storing is low; 

(e) a quantization multiplier for multiplying said usual rate of data 
compression by a second constant for each sector in which said 
visual complexity is high and said predicted bit rate cost for 

15 digitally storing is high; and 

(f) a quantization equalizer for maintaining said usual rate of data 
compression at a nearly constant value for all sectors in said video 
frame if an absolute value of a predicted increase in an overall bit 
rate cost for digitally storing all sectors in which said predicted 

20 visual complexity is low and said predicted bit rate cost for 

digitally storing is low is greater than an absolute value of a 
predicted decrease in an overall bit rate cost for digitally storing all 
sectors in which said predicted visual complexity is high and said 
predicted bit rate cost for digitally storing is high. 
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